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ABSTRACT 

exposure to a variety of instructional technologies prior to taking a 
required instructional technology course. The purpose was to identify how 
frequently students had used various technologies before entering the 
required course and to identify longitudinal changes and/or trends in 
experience levels. Participants were preservice and inservice teachers 
attending a mid-sized, public, southern state university. Between 1995-1998, 
during the first class meeting, 606 undergraduate and graduate students 
enrolled in different sections of required introductory instructional 
technology courses completed surveys indicating how frequently they had 
utilized various instructional technologies prior to beginning the course. 
Data analysis indicated that students had more experience with word 
processing than any other technology upon entry, and they seldom or never 
used spreadsheets and databases. Experience with information tools (e.g., 
e-mail, CD-ROM, and the Internet) ranged from seldom to often. By 1998, 
students reported having more prior experience with e-mail, CD-ROM, and the 
Internet than they did in 1995. Degree of prior experience varied 
significantly between groups over time and did not always move in a positive 
direction. (Contains 17 references.) (SM) 
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Developing Instructional Technology Curricula for Pre-Service & In-Service Teachers: 

A Longitudinal Assessment of Entry Skills 
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The importance of and need for P-12 educators with strong instructional 
technology skills has been well documented (U. S. Congress OTA, 1995). Continued 
emphasis on the development of critical thinking and problem-solving skills in P-12 
students, as well as the rapid expansion of technology-based information resources, 
have increased the demand for technology-fluent teachers. However, P-12 
administrators continue to complain about the difficulty of finding teachers with the 
required technology expertise (Ketcham, 1996; Ponessa, 1996). Particularly distressing 
are reports that many Colleges of Education graduate pre-service teachers who lack 
basic technology skills. Few new teachers are prepared to effectively integrate 
technology into classrooms and other learning environments to support student 
learning (Barksdale, 1996; Bentley, 1994; Northrup, 1996; Roblyer, 1994; U. S. Congress 
OTA, 1995; Wild, 1996). 

In an attempt to address these concerns, the National Council for the 
Accreditation of Teacher Education (1992) and the International Society for Technology 
and Education (1993) have identified standards for pre-service teacher technology 
competencies. Roblyer (1994) indicates that at least 50% of all teacher preparation 
programs require undergraduate pre-service teachers to enroll in at least one 
instructional technology course. Yet, complaints persist that new teachers are 
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unprepared to use technology. Grau (1996) reports that new teachers' inability to use 
technology effectively may result from the design of technology preparation courses. 
Northrup (1996) suggests that teachers' technology needs have actually shifted from 
basic productivity skills to the more sophisticated technology integration skills. Friske 
(1996) writes that it may be time to change the pre-service technology standards to 
extend beyond the basics. 

Increasing the technology standards for teachers does not negate the need for 
basic technology skills. Teacher preparation programs must ensure that pre-service 
teachers possess these competencies as well as technology integration skills. Even 
though students entering teacher education programs today have had increased 
exposure to computers and computer software at elementary and secondary schools, 
such infusion of technology has not been done in a uniform fashion across school 
districts or even within districts. Differences in students' exposure to, and attitude 
toward, computer technology may also be dependent upon the students' socio- 
economic status, or, in the case of returning students, previous work experience. The 
question becomes: "Do potential teachers entering teacher education programs today 
have basic technology skills?" Or must these skills be taught in addition to advanced 
application and integration techniques? 

Sheffield (1996) surveyed 772 pre-service teachers between 1991-1995 to 
determine their proficiency with basic word processing, database, and spreadsheet 
programs. While the students from the 1994-1995 year had more word processing 
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experience than those from the preceding three years, the overall results revealed that 
entry-level pre-service teachers (mainly juniors) had minimal experience with basic 
productivity software applications (word processing, database, spreadsheet). However, 
Sheffield suggests that the trend of increased experience that surfaced with the 1994-95 
sample may continue, and, if so, the curriculum of basic instructional technology 
courses could shift to more advanced topics such as integrating technology into the 
curriculum. 

This study followed up on Sheffield's research and examined recent (1995-1998) 
education students' exposure to a variety of instructional technologies prior to taking a 
required instructional technology course. The purpose of the research was to identify 
how frequently students had used various technologies before entering a required 
instructional technology course, and to identify longitudinal changes and/ or trends in 
the experience levels. The implication of these findings for the development of 
instructional technology curricula for pre-service teachers was also examined. 



Method 

Pre-service and in-service educators at a middle sized, public, southern state 
university (enrollment approximately 8800) with one of the nation's largest initial 
teacher preparation programs (Sethna, 1996) participated in the study. Undergraduate 
and graduate students enrolling in sections of required introductory instructional 
technology courses taught by the researchers between fall quarter, 1995 and winter 
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quarter, 1998 completed surveys indicating how frequently they had utilized a variety 
of instructional technologies prior to the beginning of the course. 

Six hundred and six students, representing the total enrollment of sixteen 
sections of instructional technology courses taught by the researchers between fall, 1995 
and winter, 1998 completed surveys. Students were enrolled in one of four courses: a 
technology/ curriculum course required for secondary education/ certification majors; a 
technology/ curriculum course required for P-12 education/ certification majors; or an 
introductory instructional technology or computer utilization course required for 
elementary education/ certification majors, middle grades education/ certification 
majors, and other undergraduate and graduate education students. The surveys were 
administered during the first class meeting by the researchers. Students were informed 
that the survey was a needs assessment instrument and that the results of the study 
would be used to help shape the course. Surveys were also administered to students 
who added the class following the first class meeting. Table 1 summarizes the number 
of surveys administered each quarter. 



Table 1 

Survey Responses 





Fall 

1995 


Winter 

1996 


Spring 

1996 


Summer 

1996 


Fall 

1996 


Summer 

1997 


Fall 

1997 


Winter 

1998 


Total 


Under- 

grads 


57 


70 


21 


27 


62 


22 


64 


48 


371 


Grads 


36 


16 


28 


39 


21 


37 


32 


26 


235 


Total 


93 


86 


49 


66 


83 


59 


96 


74 


606 
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The one-page survey collected student enrollment level (undergraduate or 
graduate), and required respondents to indicate how frequently they had used a variety 
of 14 old and new technologies such as word processing, spreadsheets, databases, e- 
mail, the World Wide Web (WWW), CD-ROM, etc. A 5-point Likert scale (l=never, 
2=seldom, 3=sometimes, 4=often, 5=a great deal) was used to collect student responses. 
Technologies included on the survey were identified after a review of instructional 
technology literature (ISTE, 1993; NCATE, 1992), instructional technology texts 
(Heinich, Molenda, Russell, 1993; Kemp & Smellie, 1994; Teague, Rogers & Tipling, 
1994), and needs assessment instruments used in similar courses at two other 
universities. The survey was finalized after several instructional technology instructors 
at the originating institution reviewed it. 



Results 

Survey responses were compiled and entered into SPSS for statistical analysis. In 
addition to the skill frequency responses, the quarter of enrollment in the instructional 
technology course and undergraduate or graduate status of the student were also 
entered. Since changes in student entry skills over time were the focus of the study, the 
data was analyzed using quarter of enrollment and student level (undergraduate or 
graduate) as independent variables. Six instructional technology skills were selected for 
analysis in this study. The three productivity tools; word processing, spreadsheet, and 
database; examined by Sheffield (1996) were targeted, as were three communication 
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and information access tools; CD-ROM, e-mail, and WWW. Thus the dependent 
variables for this study were student frequency responses on these six instructional 
technology skill items. 

The mean and standard deviation for each of the six dependent variables were 
computed for the undergraduate and graduate students each quarter. These are 
reported in Table 2. Line graphs showing the changes in means over time for skill 
frequencies are included in Appendix A. 

Table 2 



Mean Scores for Student Reported Prior Instructional Technology Experience 





Fall 

1995 


Winter 

1996 


Spring 

1996 


Summer 

1996 


Fall 

1996 


Summer 

1997 


Fall 

1997 


Winter 

1998 


Undergraduate 


M 


3.702 


3.500 


3.381 


3.222 


3.565 


4.182 


3.813 


3.500 


Word Processing 


(SD) 


(1.117) 


(1.164) 


(1.532) 


(1.281) 


(1.276) 


(1.097) 


(1.052) 


(1.111) 


Graduate 


M 


4.000 


3.688 


3.571 


3.538 


4.048 


3.676 


4.219 


3.654 


Word Processing 


(SD) 


(1.287) 


(1.078) 


(0.879) 


(1.120) 


(1.161) 


(1.248) 


(0.792) 


(1.384) 


Undergraduate 


M 


2.035 


1.629 


1.905 


1.593 


2.177 


2.364 


2.047 


1.667 


Database 


(SD) 


(1.117) 


(0.966) 


(1.091) 


(0.747) 


(1.064) 


(1.293) 


(0.950) 


(0.953) 


Graduate 


M 


2.278 


1.813 


1.679 


1.897 


2.667 


1.946 


2.563 


2.154 


Database 


(SD) 


(1.233) 


(0.834) 


(1.188) 


(1.071) 


(1.390) 


(1.053) 


(0.948) 


(1.317) 


Undergraduate 


M 


1.877 


1.443 


1.857 


1.481 


1.919 


2.000 


1.766 


1.604 


Spreadsheet 


(SD) 


(1.103) 


(0.673) 


(0.964) 


(0.802) 


(1.135) 


(1.195) 


(0.792) 


(0.736) 


Graduate 


M 


2.639 


1.563 


1.500 


1.846 


2.905 


2.000 


2.469 


2.000 


Spreadsheet 


(SD) 


(1.457) 


(0.814) 


(0.923) 


(1.014) 


(1.546) 


(1.179) 


(1.164) 


(1.200) 


Undergraduate 


M 


2.158 


1.957 


2.476 


1.815 


2.258 


2.773 


2.766 


2.313 


CD-ROM 


(SD) 


(1.192) 


(1.290) 


(1.365) 


(1.039) 


(1.330) 


(1.541) 


(1.377) 


(1.371) 


Graduate 


M 


2.194 


2.188 


2.393 


2.769 


2.619 


3.216 


3.094 


3.500 


CD-ROM 


(SD) 


(1.117) 


(1.471) 


(0.994) 


(1.224) 


(1.465) 


(1.417) 


(1.329) 


(1.421) 


Undergraduate 


M 


2.298 


1.357 


1.810 


1.630 


3.177 


2.409 


3.203 


2.458 


Email 


(SD) 


(1.281) 


(0.723) 


(0.873) 


(0.967) 


(1.477) 


(1.736) 


(1.471) 


(1.414) 


Graduate 


M 


2.361 


1.563 


1.607 


1.641 


2.810 


2.568 


3.063 


3.154 


Email 


(SD) 


(1.437) 


(1.263) 


(1.166) 


(0.873) 


(1.806) 


(1.405) 


(1.366) 


(1.759) 


Undergraduate 


M 


2.298 


1.214 


1.667 


1.630 


3.161 


2.682 


3.594 


2.833 


WWW 


(SD) 


(1.281) 


(0.720) 


(0.966) 


(0.967) 


(1.231) 


(1.555) 


(1.269) 


(1.243) 


Graduate 


M 


2.361 


1.375 


1.321 


1.744 


2.667 


2.486 


3.250 


2.769 


WWW 


(SD) 


(1.437) 


(0.885) 


(0.863) 


(1.044) 


(1.390) 


(1.346) 


(1.107) 


(1.531) 
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A simple factorial Analysis of Variance (ANOVA) was used to determine if there 
was any significant difference in the prior technology experiences reported by the 
different groups of students. An ANOVA was run for each dependent variable using 
the two independent variables as factors. Because six dependent variables were 
examined, the alpha level for each of the six ANOVAs was chosen by using the 
Bonferoni Inequality. To maintain a family wise Type I error rate of .05, the probability 
of making a Type I error for each of the six tests had to be less than .0083 (.05 divided by 
number of ANOVAS) for the results to be considered significant. The ANOVA tables 
are included in Appendix B. Significant effects (p<.008) for quarter of enrollment were 
found for database (p=.000), spreadsheet (p=.000), CD-ROM (p=.000). Email (p=.000), 
and WWW (p=.000). Significant effects for student level were found for spreadsheet 
(p=.000), and CD-ROM (p=.000). No significant interaction effects were detected. 

A number of cautions must be kept in mind when reviewing this data. First, the 
researchers did not teach any of the instructional technology introductory courses 
during the winter and spring 1997 quarters, so there was a two-quarter gap in the data 
collection process. Second, due to enrollment patterns, the cell sizes are not equal and 
this constitutes a threat to the ANOVA. Finally, some of the standard deviations for the 
frequency means in each cell are large and this, too, threatens the ANOVA. However, 
some interesting patterns did arise as these results were reviewed. 
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Discussion 

The purpose of this study was to: 

• identify how frequently students had used various technologies 
before entering a required instructional technology course, and 

• identify longitudinal changes and or trends in students' experiences 
prior to entering such a course. 

Review of the frequency means for each of the dependent variables examined in 
this study showed that education students at this institution consistently had more 
experience with word processing than any other technology tool when they entered 
introductory instructional technology courses (see Figure 1, Appendix A). Both 
undergraduate and graduate students indicated that they used word processing 
"sometimes" or "often" before beginning the course. However, students generally 
indicated that they "seldom" or "never" used spreadsheets and databases (see Figure 2, 
Appendix A). Experience with information tools such as e-mail, CD-ROM, and the 
WWW fell in the middle ranging from "seldom" used to "sometimes" or "often" (see 
Figure 3, Appendix A). It is important to note that the standard deviations for many of 
the group means were large. 

Students enrolling between fall 1995 and summer 1996, indicated that word 
processing was the only technology they used at least "sometimes." In the fall of 1996, 
in addition to graduate and undergraduate word processing exposure, undergraduates 
indicated they used e-mail and the WWW, at least "sometimes." Graduates indicated 
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they used CD-ROMs at least "sometimes" during the summer of 1997, along with the 
undergraduate and graduate at least "sometimes" use of word processing. Fall 1997 
students continued to use word processing, but graduates reported they at least 
"sometimes" used CD-ROM, e-mail, and the WWW, while undergraduates at least 
"sometimes" used e-mail and the WWW. All students used word processing at least 
"sometimes" in winter 1998, and graduate students used CD-ROM and e-mail at least 
"sometimes." These findings suggested that students enter introductory technology 
classes today with more prior exposure or experience to some technologies than their 
counterparts had in 1995. However, these increased experiences were limited to 
information access and communication technology applications, not production tool 
technologies such as database and spreadsheet. 

The analysis of variance for word processing indicated no significant difference 
in students' word processing experience prior to enrolling in a required technology 
course when viewed by quarter of enrollment or undergraduate/ graduate student 
level. Even though the differences were not significant, a review of the means 
(Appendix A, figure 2) showed a varied pattern of increases and decreases, although 
the fall quarter means were generally highest. For the most part, the undergraduate 
and graduate means followed a similar pattern. It should also be noted that the 
standard deviations for these means were frequently over 1 (on a 5-point scale). 

A significant difference in prior database and spreadsheet experience was 
detected by the analysis of variance for quarter of enrollment. However, the differences 




Kirby & Schick, AERA 1998 



10 



were not longitudinally linear. The fall quarter database and spreadsheet scores were 
always highest, followed by drops in database and spreadsheet scores during the other 
quarters (see Appendix A, Figure 2). This may suggest that students' prior experiences 
varied according to their specific major. For instance, secondary education majors 
(math, science, language arts, social studies, and business education) always took their 
technology course in the fall quarter. These students may have been more likely to use 
spreadsheet or database applications than P-12 majors (art, music, foreign language, 
and P.E.) who took the course in the winter quarter. Furthermore, the graduate mean 
responses for both spreadsheet and database were usually higher than the 
undergraduate mean responses for these applications, although the difference was only 
significant for the spreadsheet application. Again, the standard deviations for many of 
the spreadsheet and database group means were over 1 (on a 5 point scale). 

The analysis of variance for all three information access and communication 
technologies showed significant differences by quarter in students' prior experience. In 
all three information technology applications, an overall increase in prior experience 
between fall, 1995 and winter, 1998 could be seen from the means graph (Appendix A, 
Figure 3). However, it should be noted that there were occasional drops in means (i.e., 
e-mail and WWW exposure between fall, 1995 and winter, 1996; undergraduate CD- 
ROM use between spring and summer, 1996; e-mail and WWW experience between fall, 

1996 and summer, 1997; graduate and undergraduate WWW experience between fall, 

1997 and winter, 1998; and undergraduate e-mail and CD-ROM experience between 
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fall, 1997 and winter, 1998). These drops may have been due to differences in student 
majors (Secondary education, P-12, etc.) as suggested in the discussion about 
spreadsheet and database experience. 

Student level (graduate or undergraduate) was a significant source of difference 
for prior CD-ROM experience. Although graduate and undergraduate students 
occasionally reported similar prior experience levels for CD-ROMs (fall, 1995; spring, 
1996), more frequently the graduate means were higher by about 0.5 (out of 5). During 
summer 1996, and winter, 1998 the graduate means were higher by an average of 1.0 
(out of 5). 

Many of these results echoed Sheffield's (1996) findings about pre-service 
teachers' technology experiences. Sheffield found that 1991-1995 pre-service teachers 
had more word processing experience than other technology applications skills, and 
that was also true for the 1995-1998 students investigated in this study. However, while 
Sheffield found that the 1991-1995 students had minimal experience with word 
processing, database, and spreadsheet applications; this study found that the 1995-1998 
students had some degree of word processing experience, although their prior 
experience with spreadsheet and database continued to be minimal. This study also 
suggested that students' prior experience with information access technologies (CD- 
ROM, e-mail, and WWW) had increased so that most students had some degree of 
exposure to these technologies prior to entering introductory technology courses. 
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Implications 

The findings of this study generally supported Sheffield's prediction that pre- 
service teachers' technology experience prior to enrolling in introductory instructional 
technology courses may be increasing. The study showed that most education students 
had some experience with word processing and basic information access and 
communication technologies such as e-mail. However, it should be noted that the 
degree of prior experience varied significantly between groups by quarter and did not 
always move in a positive direction. Also, the standard deviations for reported prior 
experiences were frequently large suggesting that degree of prior experience varied 
within groups. While this study did not examine the effect major area of concentration 
had on prior experience, the preliminary findings suggested a hypothesis that a 
student's major may be a factor in the level of prior technology experience. If this is the 
case, then the curricula may need to be tailored to individual major cohort groups. The 
study also showed that database and spreadsheet experience continued to be minimal. 

There are several curricular implications of the study. First, some students may 
be entering instructional technology courses with more experience in word processing 
and information access and communication applications than their counterparts had in 
the past. In these cases, instructors may be able to address higher level utilization and 
integration skills without covering basic skills. For information access technologies, 
courses might focus on evaluating Internet resources as well as exploring ways to 
effectively integrate the resources into the curriculum. For most students, basic 
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instruction in productivity skills other than word processing will still be required. 

However, instructors should not be too quick too assume that increased 
experience or exposure to technology applications translates into increased skills. 
Students who have frequently used technology applications such as word processing 
may be using the computer as a typewriter and may be unaware of or lack proficiency 
in word processing tools such as cutting, pasting, tables, columns, etc. Students with a 
good deal of Internet exposure may be experienced with using the Netscape or Explorer 
browsers, but may have little experience in effective Internet search strategies using 
search engines. 

Furthermore, the large standard deviations within groups, and the varying 
directional changes in means between groups suggests that skills may vary in both 
positive and negative directions over time and within groups. While the suggestion 
that skill levels may vary according to subject major needs further research, if this is the 
case, using a needs assessment to determine appropriate instructional technology 
content for individual classes and students is critical. Different levels of instruction 
may need to be provided within one class. However, contrary to Sheffield s suggestion, 
we would recommend that integration of technology into the classroom be addressed at 
all levels of instruction. 

Longitudinal research should continue to monitor pre-service teachers' 
technology skills to provide insight into general trends and changes for the purpose of 
curriculum development. This study examined students' prior technology experience 
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looking at the factors of time and student level. Future research should also look at 
student major area of concentration as a potential factor that can affect technology 
experience. 
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Appendix A 
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Figure 1. Prior experience means by technology, quarter of enrollment, & student level. 
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Figure 2. Prior productivity technology experience means by technology, quarter of 



enrollment, & student level. 
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Appendix A (continued) 
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Figure 3. Prior information technology experience means by technology, quarter of 
enrollment, & student level. 
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Appendix B 



Analysis of Variance for Prior Technology Experience 



Source of Variation 


df 


MS 


F 


E 


Word Processing (Main) 


8 


3.539 


2.611 


.008 


Enrollment Quarter 


7 


3.462 


2.554 


.013 


Student Level (U/G) 


1 


4.582 


3.381 


.066 


Interaction 


7 


1.354 


.999 


.431 


Database (Main) 


8 


4.503 


3.956 


.000* 


Enrollment Quarter 


7 


4.562 


4.008 


.000* 


Student Level (U/G) 


1 


4.888 


4.295 


.039 


Interaction 


7 


1.758 


1.545 


.149 


Spreadsheet (Main) 


8 


7.748 


7.252 


.000* 


Enrollment Quarter 


7 


6.402 


5.992 


.000* 


Student Level (U/G) 


1 


17.316 


16.207 


.000* 


Interaction 


7 


2.883 


2.698 


.009 


CD-ROM (Main) 


8 


11.930 


7.001 


.000* 


Enrollment Quarter 


7 


8.798 


5.163 


.000* 


Student Level (U/G) 


1 


23.424 


13.746 


.000* 


Interaction 


7 


3.065 


1.799 


.085 


E-mail (Main) 


8 


23.279 


13.365 


.000* 


Enrollment Quarter 


7 


26.599 


15.271 


.000* 


Student Level (U/ G) 


1 


.350 


.201 


.654 


Interaction 


7 


1.687 


.969 


.453 


WWW (Main) 


8 


32.421 


22.934 


.000* 


Enrollment Quarter 


7 


36.335 


25.703 


.000* 


Student Level (U/G) 


1 


2.394 


1.693 


.194 


Interaction 


7 


.945 


.669 


.699 



*significant for p<.008 
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